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ABSTRACT 

A study determined the levels of comprehension 
generated by questions accompanying the stories in new basal readers* 
A sample of 200 questions, 100 randomly selected from each of the 2 
most widely used third-grade series (the Scott Foresman series 
"Celebrate Reading" and the McGraw-Hill series "A New View"), were 
analyzed for the number of literal and above literal-level questions* 
Both basal series were published in 1995. Results indicated that 71% 
of the questions dealt with inferential comprehension while only 12% 
fell into the literal category* Results were similar when questions 
from each series were analyzed separately. Findings suggest that 
previous trends of a preponderance of literal-level questions has 
been reversed — students answering questions that accompany the 
stories in the new basals engage in higher levels of comprehension 
rather than mere memorization. (Contains 12 references and 2 tables 
of data.) (RS) 
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The New Basal Readers: What Levels of Comprehension Do They Promote? 



Background and Statement of the Problem 



A major goal of reading instruction, according to outcomes stated at both state 
and local levels, is to teach students to perform higher-level comprehension tasks such 
as acquiring and systematically arranging information, distinguishing relevant from 
irrelevant information, deciding how to use data, detecting cause and effect 
relationships, and the like. In recent years, such sophisticated goals enhanced the 
notion that reading is a thinking process that cannot be taught effectively through 
drill and repetitive practice alone. Concerning the importance of critical reading 
skills. Leu and Kinzer (1992) wrote: "In reading ... the learner's major objective is to 
develop, refine, and use higher-level text thought processes - ultimately, to 
comprehend" (p. 2). 

Indeed, examination of reading instructional materials reveals that higher- 
level comprehension is paramount among learner outcomes. Additionally, 
investigative commissions such as the National Assessment of Educational Progress 
(NAEP, 1987) and Becoming a Nation of Readers (Anderson, 1985) have placed heavy 
emphasis on critical thinking in reading. 

The realization of this goal, however, is questionable given the fact that 

elementary students primarily receive decoding and literal comprehension 

opportunities with basal readers. More often than not, teachers in elementary schools 

follow the dictates of published reading programs that emphasize isolated skill 

activities targeting decoding, vocabulary, and literal comprehension (Duffy & 

McIntyre, 1982; Durkin, 1979; Palmer, 1982). As Samuels and Farstrup (1992) stated: 

Often the teacher's role is primarily that of a technician who 
follows directions and prescriptions, rather than a decision- 
maker who engages in substantive pedagogical maneuvering 
in response to students' needs. However, drill and practice 
instructional models are inadequate for the new 
comprehension curriculum. This is especially true in a 
technological society - a society that will increasingly value 
workers who can solve problems over those who can follow 
prescribed routines. It is no longer good enough to have 
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students answer literal questions and memorize isolated skill 
responses, (pp. 171-172) 

It is well established in the literature that elementary teachers rely heavily on 
basal readers and the accompanying lesson plans for reading instruction. Although 
implementation differs among teachers, research demonstrates that published 
programs greatly affect classroom practices (Diederich, 1973; Durkin, 1984). 

The heavy reliance on basals would not pose a problem with reading instruction if 
they contained effective lessons that promoted higher-level comprehension. 
Unfortunately, published reading programs have remained virtually unchanged since 
the 1940s. While the graphics and attractive packaging have improved since the early 
days, basals still overemphasize decoding and literal level comprehension at the 
expense of higher-level comprehension. The most traditional lessons employ the 
Directed-Reading Activity (DRA) format, which is skill-driven. Specifically, the 
objective of each lesson is to learn an isolated skill (which may or may not be relevant 
to understanding the story) rather than to comprehend a particular type of discourse. 
While students are held accountable for learning the skill through independent 
workbook practice and/or end-of-unit tests, they are seldom held accountable for the 
comprehension of the reading selection — except through questioning. Moreover, the 
literature indicates that these questions evoke recall of story details and that teachers 
tend to rely on these questions during the teaching of basal lessons ( Winograd, 
Wixson, & Lipson, 1989). 

In the mid-1990s, a dramatic shift occurred among most major publishing 
companies who for so many years had emphasized a skills framework. This shift can 
be attributed to many factors. The continued decline in higher-level reading skills 
and achievement in the United States (Flanagan, 1976; NAEP, 1985) accompanied by 
the increased interest in whole language prompted elementary teachers to search for 
commercially-prepared materials that incorporated strategies such as: shared reading; 
guided reading; language experience; process writing; read-aloud; thematic teaching; 
and the like. These strategies emphasize meaning instead of isolated drill of the 
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lower-level components of reading and initiate the teaching act with focus on the 
entire story. The excitement among teachers created by these and other comprehension- 
based techniques prompted basal publishers to analyze products, which had not 
changed in decades, and modify their total philosophy in an effort to match the 
demands of the market. In short, the overcrowded textbook publishing market 
underwent a metamorphosis unparalleled in reading basal history. In the new basals 
of 1994-1995, whole language methods permeate lesson plans and encourage teachers 
to utilize the reading-writing connection, think-alouds, read-alouds, thematic units, 
and most importantly - children's literature. Moreover, publisher representatives 
now portray potential adopted texts as tools of whole language while making the 
decree: "We listened to teachers and responded to what they wanted." The amount of 
ancillary materials that accompany the new basals nearly doubled and included: big 
books, chapter books, journal notebooks, interactive videodiscs, and so forth. This 
move surprised a potential market that once relied on basals with diluted vocabulary 
and predictable workbook drills. 

At first exposure, these materials impressed teachers who desired more focus 
on meaning and real-reading experiences. Excerpts of authentic stories in the 
children's literature "anthology" (the new term for basal) included original portions 
of such notable classics as Charlotte's Web . While new strategies and materials do 
exist among the new basals, one component remains unchanged: the preponderance of 
questions interspersed throughout the basal story. Moreover, research in the area of 
questioning indicates that teachers rely heavily on story-related questions included in 
the lesson plans that accompany the basal (Reutzel & Hollingsworth, 1991) and 
historically these questions tend to emphasize literal or recall-only comprehension. 
Reading publishers proudly proclaim a renewed emphasis on meaning-centered 
instruction, yet teachers must ask: Have the new basals changed their approach to 
questioning and become sensitive to the critical comprehension movement? 
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Analyzing New Basal Readers 

In an effort to determine the levels of comprehension generated by questions 
accompanying the stories in new basal readers, an analysis of the questions was 
completed. The research examined the extent to which the story-related questions 
emphasized higher-level comprehension. Further, this study sought to examine if a 
statistically significant difference existed between the number of literal and above- 
literal-level questions (i.e., inferential and evaluation). It was hypothesized that there 
would be no significant difference between the frequencies of story-related questions 
classified at the literal level and the above-literal-level of comprehension (i.e., 
inferential and evaluation combined). 



Research Procedures 



Guided reading stories that accompany the new basal readers for grade 3 were 
obtained for the actual classification study. The sample consisted of 200 questions, 100 
randomly selected from each of the two most widely used third-grade series from 
Alabama and Texas combined. Data from both state departments revealed that the 
Scott Foresman (1995) series entitled: "Celebrate Reading" and the /McGraw-Hill 
(1995) series entitled: "A New View" were the most widely adopted series in the two 
states combined as of Spring 1996. The 200 questions were randomly selected prior to 
the classification and extracted from all interrater reliability exercises and training 
discussions. 

After extensive training in question classification with the levels of literal, 
inferential, and evaluative comprehension and obtaining an interrater agreement of 
.96 on a sample of questions not selected for the actual classification study, three raters 
independently classified 200 questions, 100 from each series according to three 
comprehension categories: literal, inferential, and evaluation. 
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Findings and Conclusions 

To represent the data, answer the research question, and test the proposed 
hypothesis, the study utilized frequency distribution tables and the chi-square 
goodness-of-fit analysis. The chi-square goodness of fit analysis determined if the 
observed frequency differed from that expected by chance. 

The research question asked: What levels of questions are promoted by 
questions that accompany both basal series for guided reading? In order to answer 
this research question, the following null hypothesis was tested: There is no 
significant difference between the frequencies of questions classified at the literal- 
level and above-literal-level. 

Each question randomly selected for classification was rated independently by 
three trained raters. The results of the ratings were compared and the question was 
placed in the category ascribed by the majority of raters. In the actual classification 
study, the raters reached total agreement on 93% of the questions. Table 1 reports the 
frequency distribution of the results of the classification for both series combined. 



Table 1 


FREQUENCY AND PERCENTAGE OF STORY-RELATED QUESTIONS FOR THE 
SCOTT, FORESMAN AND MACMILLAN/ MCGRAW-HILL SERIES COMBINED 
ACCORDING TO THE LEVELS OF COMPREHENSION 


Comprehension Level 


n 


% 


Literal 


25 


12 


Inferential 


142 


71 


Evaluative 


33 


17 




200 


100 



Note. Percentages are rounded to nearest whole numbers. 
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Inspection of these data indicate that the majority of story-related 
comprehension questions that accompany the selected basal series were classified in 
the inferential category. The overall frequency and percentage tabulation reveals that 
of all 200 questions randomly selected for analysis, 142 (71%) dealt with inferential 
comprehension. The second largest percentage of questions was classified as 
evaluation, 33 (17%); only 25 (12%) fell into the literal category. 

Although these data represent a combination of both series, comparisons 
between the two selected series yield significant similarities that should be noted. 
When analyzed separately, the two series yield remarkably similar results that 
highlight a dramatic change in the new basals. The Macmillan/ McGraw-Hill (1995) 
series contained 71 inferential comprehension and 16 evaluation items for a total of 
87% above-literal comprehension questions. It is significant to note that only 13% of 
the comprehension questions emphasize literal recall. These data are represented in 
Table 2. 





Table 2 




INDIVIDUAL FREQUENCY AND PERCENTAGE OF BOTH TEXTBOOKS' EMPHASIS 

ON LEVELS OF COMPREHENSION 


Basal Reading Series 


Literal 
n % 


Inferential Evaluation 

n % n % 


Total 


Macmillan / McGraw-Hill 


13 (13) 


71 (71) 16 (16) 


100 


Scott, Foresman 


12 (12) 


71 (71) 17 (17) 


100 



Note. Percentages are rounded to nearest whole numbers. 



The Scott, Foresman (1995) series demonstrated a near perfect replication with 
respect to the three comprehension categories. An analysis of these data in the above- 
literal-categories indicate that 71 questions were judged as inferential and 17 items 
were judged as evaluation resulting in a total of 88% above- knowledge-level 
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questions for this series. Only 12% of the questions that accompany the story were 
categorized as literal. 

Of major note is the finding that two of the most widely-used reading series 
(grade 3) from Texas and Alabama produced a uniform emphasis on the higher-levels 
of comprehension: inferential and evaluation. Obviously, the selected basals 
promote inferencing and evaluation rather than a preponderance of literal-level 
comprehension questions. 

To test the null-hypothesis, a chi-square goodness-of-fit analysis was conducted 
to determine if the observed differences between the total frequencies of the 
combined textbooks' comprehension questioning (literal vs. above-literal-level) 
departed significantly from those expected by chance. Because the chi-square value of 
32.23 exceeds the critical chi-square value beyond the .001 level, the null hypothesis is 
rejected. This statistic indicates that the distribution of total frequencies for the literal 
and above-literal-level comprehension questions departed significantly from a 
distribution based on chance alone. Therefore, the selected basals include 
significantly more above-literal-level comprehension questions (i.e., inferential and 
evaluation). 



Implications 

It should be noted that the new basals consist of three main parts related to the 
guided-reading plan for each story: (a) Building Background Knowledge and 
Motivation, (b) Guided Reading of the Story, and (c) Story Extension. This research 
investigated comprehension questions included in teacher's editions for use before, 
during, and after reading. Questions conceived for use in story-extension were not 
analyzed. 

Additionally, a significant change has occurred in the publishing industry. 
Regarding reading basals... the number of series available for adoption as a core basal 
has diminished significantly due to such factors as the consolidation of companies or 
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the company discontinuing publication in the area of reading. This research selected 
for study two of the five most widely used basals available for adoption in the United 
States and, most importantly, selected for study two of the most widely used basals 
for the states of Alabama and Texas combined. In short, the sample used in this study 
is highly representative of the basals available for adoption and used in today's 
classrooms. 

This study reveals an exciting and surprising shift in the comprehension 
emphasis contained in the new basals. Not so long ago, these same publishers 
included a preponderance of literal-level questions related to the story. Today, the 
trend has reversed dramatically, and these data show that the new basals include not 
only meaning-centered teaching strategies, but they now include meaning-centered 
questioning that correlate with such methods. 

Research verifies that the type of question used by teachers strongly affects 
what children learn to think about while reading. If literal details receive the 
emphasis in a teacher's questioning, then children learn to attend to details as they 
read (Anderson, 1985). Therefore, repeatedly asking the same type of question cues 
students to focus on details needed to answer the question, and they locate their 
attention accordingly. On the other hand, studies (e.g., Pearson, 1985) show that 
making inferences and evaluations result in improvement in critical thinking without 
loss, and even a gain, in higher-level comprehension because active manipulation of 
literal events enhances retention; students must integrate literal details and integrate 
them into their world or little comprehension occurs. 

Data in this study imply that students, while answering the questions that 
accompany the story in the new basals, engage in higher-levels of comprehension 
rather than mere memorization. Further, these data indicate that the new reading 
basals do in fact include opportunities for students to experience higher-levels of 
comprehension before, during, and after story reading. 

For years publishers and the on-site representatives have made lofty claims 
regarding critical reading only for teachers and students to discover that few 
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experiences beyond the literal-level exist. The new basals, for the first time, validate 
such claims with a renewed emphasis on inferencing and evaluation. 

While it is hoped that no teacher will rely exclusively on a single material (i.e., 
the new basal) to meet the complex needs of today's elementary students, novice 
teachers and those who do use a basal exclusively should find these results 
encouraging. These results should prompt teachers to examine closely each lesson 
plan and identify such questions. Most importantly, teachers must use higher-level- 
questions (as well as literal-level questions) to provide students with the opportunity 
to infer and evaluate. With the new basals, this opportunity exists for the first time in 
recent memory. 
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